Skip to content

fix(core): a lone failing cached call no longer kills the process (#670) - #676

Merged
rejifald merged 1 commit into
mainfrom
claude/cache-inflight-unhandled-rejection
Aug 6, 2026
Merged

fix(core): a lone failing cached call no longer kills the process (#670)#676
rejifald merged 1 commit into
mainfrom
claude/cache-inflight-unhandled-rejection

Conversation

@rejifald

@rejifald rejifald commented Aug 6, 2026

Copy link
Copy Markdown
Owner

Fixes #670. A cached stitch whose vendor fails, with no concurrent caller, emitted an unhandled promise rejection — which under Node's default --unhandled-rejections=throw terminates the process. The caller had already handled the failure: .safe() returned an honest ok: false, and the program died anyway.

Reproduced first, on unpatched main

The issue's 14-line reproduction, run against main's source:

caller handled it: ok = false
packages/core/src/engine.ts:1693
            claim.fail(new Error('cache: leader run failed'));
                       ^

Error: cache: leader run failed
    at runCached (engine.ts:1693:24)
    at async execute (engine.ts:1769:9)
    at async wrapped (stitch.ts:829:26)
    at async drain (stitch.ts:526:22)
    at async consumeSafe (stitch.ts:804:21)

Node.js v24.18.1
EXIT CODE: 1

STILL ALIVE — the line after the handled call — never printed. After:

caller handled it: ok = false
STILL ALIVE
EXIT CODE: 0

The issue's severity table reproduces exactly, and every row is now zero:

shape before after
1 lone failing call 1 0
same failure, a concurrent follower present 0 0
same failure, coalesce: false 0 0
healthy vendor 0 0
20 staggered deliveries against a failing vendor 20 0

That second row is the whole reason this survived: a test exercises coalescing with a concurrent burst, and a follower's await catches the rejection by accident. A webhook backlog or a retry drain arrives staggered, where every call is its own leader.

The fix, and why this shape

One statement, in InflightCoalescer.join, attached to the shared promise where it is created:

const promise = new Promise<T>((res, rej) => { resolve = res; reject = rej; });
promise.catch(() => { /* an audience of nobody is not an error */ });

The reasoning, since "add a .catch" is exactly the move that deserves scrutiny:

It swallows nothing a follower needs. .catch() returns a derived promise, which is discarded. entry.promise — the object a follower awaits — is untouched. A follower's await claim.promise still rejects, same tick, same error identity. Verified rather than asserted: with three concurrent callers against a failing vendor, all three still receive their own honest HTTP 503 (never the cache: leader run failed sentinel) and the adapter is still called three times, before and after. What is retired is the coalescer's own liability, not the channel.

That matters because of #653. #653 asks for the leader's rejection to be shared with joiners. A smaller fix was available and rejected for this reason: resolving a sentinel instead of rejecting would remove the hazard outright and save bytes, but it deletes the rejection channel #653 would build on. This leaves it intact.

Why at construction rather than in fail(). Attaching where the promise is born makes safety structural — it does not depend on who happens to join, or when, or on the order in which fail and a late join interleave. The promise is never in an unobserved-rejection state for a single tick.

Why not "only reject when waiters > 0", the issue's other suggestion. It leaves the promise permanently pending, which trades a crash for a hang and is a worse trap for the next reader. And refs is not a reliable proxy for "someone is listening": it is decremented by aborts, so a run can have refs === 1 with a promise someone still holds. Rejecting unconditionally keeps fail()'s contract honest; the guard makes it harmless.

A note on the sentinel. claim.fail(new Error('cache: leader run failed')) never reaches a user — the follower catches it and ignores it. It is a control signal meaning "don't wait for me", not an error report. Marking that signal handled is not a loss of information.

What I deliberately did not change

LeaderClaim.promise is never read. The leader has settle/fail — the write end — and awaiting its own promise before settling would deadlock. Dropping the field is tempting and I implemented it, then reverted it: it is a public type change on the stitchapi/cache subpath (LeaderClaim is in lib/cache.d.ts's export list), and measured at 3 gzip bytes it pays for none of the budget below. That is a decision worth making on its own, not as a rider on a crash fix. The interface now carries a comment saying why the field must never be awaited.

Regression tests — three, and all three fail without the fix

Reverting packages/core/src/cache.ts to origin/main and leaving the tests in place:

× test/cache-internals.spec.ts > InflightCoalescer > a LONE leader failure is not an unhandled rejection (#670)
× test/cache.spec.ts > cache — in-process coalescing > a LONE failing leader leaves no unhandled rejection (#670)
× test/cache.spec.ts > cache — a handled failure does not kill the process (#670) > the issue's reproduction exits 0 and reaches the line after the call
   Tests  3 failed | 53 passed (56)

with the failures reading, respectively, expected [ Error: cache: leader run failed ] to deeply equal [] and expected 'handled: ok=false\n' to contain 'STILL ALIVE'. Restored: 56 passed (56).

Two in-process and one subprocess, because they prove different things:

  1. cache-internals.spec.ts — the coalescer's own contract: a lone join + fail, no engine involved.
  2. cache.spec.ts, engine level — mirrors the precedent at cache.spec.ts:339-361 (the LRU-eviction guard, which already established that fire-and-forget promises in the cache path must not surface). Written as a single sequential call, as the issue insists: a burst hides the bug, which is precisely why the existing "a leader failure is NOT shared" test a few lines above never caught it.
  3. A subprocess test. The in-process assertions prove an unhandledRejection event does not fire. What a caller experiences is the exit code, and a test runner installs handlers of its own — so an in-process test can pass while the same code still kills a real program. This one bundles src/ with esbuild (already a devDependency, already how scripts/bundle-size.mjs measures), writes the issue's reproduction, and spawns a bare node with NODE_OPTIONS stripped so the default --unhandled-rejections=throw really is in force. It asserts exit 0 and that stdout reaches the line after the call. Bundling from src rather than lib means it needs no prior pnpm build and tests the working tree. Cost: ~140 ms.

Neighbours checked

  • A coalesced follower still receives the leader's failure. 3 concurrent callers, failing vendor: ok=false/false/false, messages ["HTTP 503","HTTP 503","HTTP 503"], 3 adapter calls. Identical before and after. Failure is still not shared (each follower re-runs independently) — that is a coalesced failure is not shared, and a store silently un-pools pool: 'host' #653's subject, unchanged here.
  • A successful coalesced run is unaffected. 3 concurrent callers, healthy vendor: 1 adapter call, all three served {"hit":1}. Identical before and after.
  • The full suite: 139 files / 1491 tests, all green.

A second finding — reported, not fixed

The import { stitch } budget is charging for a chunk a consumer's bundler would split out. cache.ts's own header says it is "shipped behind the stitchapi/cache subpath so import { stitch } pulls none of it", and at runtime that is true — the engine reaches it by a lazy import('./cache') only when a stitch carries a cache block. But bundle-size.mjs measures with esbuild and no splitting, so the dynamic import is inlined and counted. Measured both ways on this branch:

gzip
single file (what the gate measures) 22026 B
code-split: main chunk 16926 B
code-split: cache-*.js 3339 B
code-split: shared chunk 2709 B

So a real bundler keeps ~3.3 KB gzip of cache code out of the initial payload of import { stitch }, and the advertised figure overstates by roughly that. I have not touched the gate — changing its methodology would move the advertised number by ~3 kB, which is a far bigger maintainer-visible decision than this fix and does not belong on a crash fix. Flagging it so it can get its own issue.

Bundle budget — raised, and the advertised figure moves

Stated up front because the gate's own comment says a raise must be deliberate.

scenario main here Δ budget headroom
whole entry 24656 B 24664 B +8 B 24.10 KB (unchanged) 14 B
import { stitch } 22015 B 22026 B +11 B 21.50 → 21.55 KB 41 B

A minimum step, not the ~0.2 KB this gate usually restores — matching #477/#524/#485 and, most recently, #675: this is a fix squeezing past a full ceiling, not a new capability, and main had run down to one byte. Sizing a larger step is the maintainer's call, not a bug fix's; I will happily widen it if you prefer the #620 philosophy ("so the next small core-path fix is not gated on a budget PR of its own"), because at 41 B and 14 B the next one certainly will be.

There is no version of this that fits under a zero-byte ceiling. The guard is ~11 gzip bytes; the cheaper alternatives were measured and rejected above on correctness grounds, not size.

The advertised figure moves — and not because of this change. 21.5 KB is both the budget and a rounding boundary (22016 B), and main measured 22015 B: one byte below both. Any core-path byte takes import { stitch } from ~21 → ~22 kB. Nine figures across the six sites under the bundle-advertised-size tether — both READMEs, the installation and principles pages, the home-page metrics component, and the docs' source blurb — propagated by hand and then verified with the tether, not assumed:

ok  bundle-advertised-size  —  all sites agree on "22, 24"
6 tether(s): 6 ok, 0 fixed, 0 drift, 0 conflict, 0 broken

Two things worth knowing about that propagation:

  • The core README's ~21 kB brotli moved with them and is more accurate for it: the whole entry's brotli is 21.6 KB, which rounds to 22, not 21. It was stale before.
  • feat(core): QUERY is a read, not a write (Refs #462) #675 raises the same budget and moves the same nine figures, so these two will conflict on bundle-size.mjs and the six doc files. Whichever lands second should re-run the tether rather than trust the merge; the doc half is a pure value swap either way. yakir.lock is not touched here — its baseline was already behind on five tethers before this branch, and check passes on site agreement.

Verification

Every gate, run to completion:

test exit 0 — core 139 files / 1491 tests passed, every package green
check:types 0 — 38 projects
check:types-d (tsd) 0
check:lint 0 — clean across 36 packages
check:contract 0 — no new violations (0 known, baselined)
check:format 0
check:changelog 0 — 23 subsections, Unreleased in Keep a Changelog order
check:docs-links 0 — 116 routes, every /docs/... link resolves
check:size 0 — against the raised budget above
check:unknown-keys 0
check:exports (attw) 0
check:release 0
yakir --tier executable 6/6 ok

Nothing failed. CHANGELOG entry under Unreleased → Fixed.

🤖 Generated with Claude Code

Adding `cache: { ttl }` to a stitch turned a handled vendor failure into
an unhandled promise rejection, which under Node's default
`--unhandled-rejections=throw` terminates the process. `.safe()` returned
an honest `ok: false` and the program died anyway; the same failure with
no `cache` block produced none.

The coalescer's leader rejects one shared promise to release its waiters.
With no concurrent caller there are no waiters, so nothing ever attached
a handler. That made it invisible in the shape a test takes — a
concurrent burst, where a follower's `await` catches the rejection by
accident — and fatal in the shape production has, a staggered backlog
where every call is its own leader. 20 staggered calls against a failing
vendor produced 20 unhandled rejections; the same 20 as a burst produced
none.

The shared promise now carries a terminal no-op handler from the moment
it is created, so being unobserved is never fatal. The handler is
attached to a derived promise and discarded, so a follower's `await`
still sees the same rejection, same tick, same error identity — the
channel #653 wants to hand failures down is untouched.

`import { stitch }` gains 11 gzip bytes, and `main` had one to spare, so
the budget takes a minimum step (21.50 → 21.55 KB). 21.5 KB is also a
rounding boundary, so the advertised figure moves ~21 → ~22 kB across the
six sites under the `bundle-advertised-size` tether — any core-path byte
would have moved it.

Closes #670

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@rejifald
rejifald merged commit 038a57b into main Aug 6, 2026
12 checks passed
@rejifald
rejifald deleted the claude/cache-inflight-unhandled-rejection branch August 6, 2026 14:22
rejifald added a commit that referenced this pull request Aug 8, 2026
…) (#717)

`verdict: { accept: [404] }` makes a declared non-2xx a successful
outcome, and the cache's store gate was `out.ok` and nothing else — so
the ABSENCE behind the 404 was written to the store. A record created a
second later stayed masked for the whole TTL, on a stitch whose only sin
was declaring that a 404 is normal.

The second consequence is worse, and it is what settles the shape of the
fix. A hit is replayed as `resultEvt(found.data, found.status, 0)`
without running the reader's `interpret`/`verdict` at all, so a stored
404 could be served as a success to a second stitch whose own verdict
rejects that status — the entry's accept list belonging to its WRITER
rather than its reader. Two stitches reach one key easily: the cache id
falls back to `cfg.name ?? cfg.path ?? 'stitch'`, so two unnamed
stitches over one URL already share a namespace (that collision is #704
§3 and stays open).

The gate now reads the status, and reads it accept-BLIND: `< 400` is
`classifyStatus`'s own threshold minus the accept term, which is exactly
the set every reader counts as a success on its own merits. Nothing a
`verdict.accept` had to rescue is durable, so the cross-stitch replay is
unreachable through this path regardless of §3.

Coalescing is deliberately untouched. `claim.settle` stays ungated,
because the coalescer belongs to the stitch that owns it and every
follower therefore shares the leader's verdict — sharing one concurrent
accepted 404 among identical callers is the live behaviour each would
have got anyway. Only the durable write, which outlives the burst and
can be read back by someone else, is held back.

Semver: a `fix`, not a break. A caller deliberately caching an accepted
404 does lose that caching and pays a live call per call — the one
visible consequence, and the CHANGELOG says so. But the entry it loses
was never sound: ADR 0003 decision 2 has it that a stored entry is
always a known-good value, and this one could be handed to a reader that
rejects it. The success VALUE a caller sees is unchanged; only staleness
and the origin-call count move, and both move toward correct.

Written inline rather than as a `ctl.storableStatus` predicate with a
`bypass:` trace event, which is what this started as. `runCached` is in
the core entry and #676 left it 14 B under budget; the predicate plus
the event measured +30 B and went red, and even the bare predicate with
no event was +21 B and still red. The inline form is +6 B on the whole
entry and +7 B on `import { stitch }`, so both budgets hold unchanged at
24.10 / 21.55 KB with 8 B / 34 B of headroom, and the advertised 24 / 22
kB figures do not move. The cost is that the skipped write is silent;
the reasoning is a comment at the gate, which costs nothing at runtime.
Sizing a budget step is the maintainer's call, not a bug fix's — #676
says so in that file, so this fix fits under the ceiling instead.

Tests: an accepted 404 no longer masks a record created after it; the
same key still misses and then caches the 200 that follows; a second
stitch whose verdict rejects 404 reaches the origin instead of being
handed the writer's entry; and a plain 200 caches and replays as before.
The first three fail on `main`.

Refs #704

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

cache: a lone failing call emits an unhandled rejection and terminates the process

1 participant